Structured illumination microscope, structured illumination method, and program

ABSTRACT

A structured illumination microscope includes a spatial light modulator containing ferroelectric liquid crystals, an interference optical system for illuminating a specimen with an interference fringe generated by making lights from the spatial light modulator interfere with each other, a controller for applying a voltage pattern having a predetermined voltage value distribution to the ferroelectric liquid crystals, an image forming optical system for forming an image of the specimen, which has been irradiated with the interference fringe, an imaging element for generating an image by imaging the image formed by the image forming optical system, and a demodulating part for generating a demodulated image using a plurality of images, wherein the controller applies an image generation voltage pattern for generating the demodulated images and a burn-in prevention voltage pattern calculated based on the image generation voltage pattern to the ferroelectric liquid crystals.

CROSS-REFERENCE TO RELATED APPLICATION

This is a Divisional Application of U.S. patent application Ser. No.15/103,646 filed on Jun. 10, 2016, which is a Continuation Applicationof International Application No. PCT/JP2014/082804, filed on Dec. 11,2014, which claims priority to Japanese Patent Application No.2013-257512, filed on Dec. 12, 2013. The contents of the aforementionedapplications are incorporated herein by reference in their entirety.

BACKGROUND

Field of the Invention

The present invention relates to a structured illumination microscope, astructured illumination method, and a program.

Background

Within microscope apparatuses, there is a super-resolution microscopethat makes possible observation exceeding the resolution of an opticalsystem.

Structured illumination microscopy (SIM) is known as one aspect of thesuper-resolution microscope, wherein a super-resolution image of aspecimen is generated by illuminating a specimen with spatiallymodulated illumination light to obtain a modulated image anddemodulating the modulated image (for example, refer to U.S. ReissuedPatent Invention No. 38,307). In this method, a light flux exited from alight source is branched into a plurality of light fluxes by adiffraction grating or the like, and a modulated image of the specimenis obtained by illuminating the specimen with an interference fringe,which is formed by making the light fluxes interfere with each other inthe vicinity of a specimen.

SUMMARY

In the structured illumination microscope described above, using aspatial light modulator that uses ferroelectric liquid crystals as adiffraction grating or the like for branching a light flux into aplurality of light fluxes is known. By applying a drive voltage to theliquid crystal element composing the spatial light modulator, the phaseof the illumination light passing through the liquid crystal element canbe modulated.

Meanwhile, in a liquid crystal display apparatus using ferroelectricliquid crystals, when a drive voltage of the same symbol is continuouslyapplied to a liquid crystal element, a phenomenon called burn-in mayoccur, where even if a voltage is applied to the liquid crystal element,it does not change to another stable condition. Because of this, in aliquid crystal display device using ferroelectric liquid crystals, therehas been proposed a method for preventing burn-in by applying thereverse voltage of the drive voltage to the liquid crystal element.

However, in the structured illumination microscope described above, whenusing a spatial light modulator that uses ferroelectric liquid crystals,there exists a problem in that the imaging time is increased compared tonot applying a reverse voltage, because the time during which thereverse voltage of the drive voltage is being applied to the liquidcrystal element is unnecessary time for the structured illuminationmicroscope. That is, in a structured illumination microscope asdescribed above, there exists a problem in efficiently preventingburn-in of the liquid crystal element when using a spatial lightmodulator that uses ferroelectric liquid crystals.

An object of an aspect of the present invention is to provide astructured illumination microscope, structured illumination method, andprogram that can efficiently prevent burn-in of a liquid crystal elementused as a spatial light modulator.

A structured illumination microscope according to one aspect of thepresent invention includes a spatial light modulator containingferroelectric liquid crystals, an interference optical system forilluminating a specimen with an interference fringe generated by causinglights from the spatial light modulator interfere with each other, acontroller for applying a voltage pattern having a predetermined voltagevalue distribution to the ferroelectric liquid crystals, an imageforming optical system for forming an image of the specimen, which hasbeen irradiated with the interference fringe, an imaging element forgenerating an image by imaging the image formed by the image formingoptical system, and a demodulating part for generating a demodulatedimage using a plurality of images, wherein the controller applies animage generation voltage pattern for generating the demodulated imagesand a burn-in prevention voltage pattern calculated based on the imagegeneration voltage pattern to the ferroelectric liquid crystals.

A structured illumination method according to one aspect of the presentinvention includes (a) illuminating a specimen with an interferencefringe generated by causing lights from a spatial light modulatorcontaining ferroelectric liquid crystals to interfere with each other,(b) applying a voltage pattern having a predetermined voltage valuedistribution to the ferroelectric liquid crystals, (c) forming an imageof the specimen illuminated by the interference fringe, (d) generatingan image by imaging the image formed in (c), and (e) generating ademodulated image using a plurality of the images, wherein an imagegeneration voltage pattern for generating the demodulated image and aburn-in prevention voltage pattern calculated based on the imagegeneration voltage pattern are applied to the ferroelectric liquidcrystals in (b).

A program according to one aspect of the present invention is a programfor causing a computer to execute (a) illuminating a specimen with aninterference fringe generated by causing lights from a spatial lightmodulator containing ferroelectric liquid crystals to interfere witheach other, (b) applying a voltage pattern having a predeterminedvoltage value distribution to the ferroelectric liquid crystals, (c)forming an image of the specimen illuminated by the interference fringe,(d) generating an image by imaging the image formed in (c), and (e)generating a demodulated image using a plurality of the images, whereinan image generation voltage pattern for generating the demodulated imageand a burn-in prevention voltage pattern calculated based on the imagegeneration voltage pattern are applied to the ferroelectric liquidcrystals in (b).

According to an aspect of the present invention, burn-in of a liquidcrystal element used as a spatial light modulator can be efficientlyprevented.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a simplified drawing illustrating the observation apparatusaccording to the first embodiment of the present invention.

FIG. 2A is a schematic drawing illustrating an example of theconfiguration of the light modulating part of the present embodiment.

FIG. 2B is a schematic drawing illustrating an example of theconfiguration of the light modulating part of the present embodiment.

FIG. 3A is a schematic drawing illustrating an example of the phase ofthe light modulated by the light modulating part of the presentembodiment.

FIG. 3B is a schematic drawing illustrating an example of the phase ofthe light modulated by the light modulating part of the presentembodiment.

FIG. 3C is a schematic drawing illustrating an example of the phase ofthe light modulated by the light modulating part of the presentembodiment.

FIG. 4 is a schematic drawing illustrating an example of the voltagepattern applied by the drive controller of the present embodiment.

FIG. 5 is a schematic drawing illustrating an example of the sequence ofthe voltage pattern applied to the light modulating part by the drivecontroller of the present embodiment.

FIG. 6 is a schematic drawing illustrating a comparative example of theapplication sequence of the voltage pattern.

FIG. 7 is a schematic drawing illustrating a modified example of thesequence of the voltage pattern applied to the light modulating part bythe drive controller of the present embodiment.

FIG. 8 is a schematic drawing illustrating the first modified example ofthe voltage pattern applied by the drive controller of the presentembodiment.

FIG. 9 is a schematic drawing illustrating the second modified exampleof the voltage pattern applied by the drive controller of the presentembodiment.

FIG. 10 is a schematic drawing illustrating the third modified exampleof the voltage pattern applied by the drive controller of the presentembodiment.

FIG. 11 is a schematic drawing illustrating the fourth modified exampleof the voltage pattern applied by the drive controller of the presentembodiment.

FIG. 12 is a schematic drawing illustrating the fifth modified exampleof the voltage pattern applied by the drive controller of the presentembodiment.

FIG. 13 is a schematic drawing illustrating a modified example of theapplication sequence of the voltage pattern applied by the drivecontroller of the present embodiment.

FIG. 14A is a schematic drawing illustrating an example of theinterference fringe imaged by the imaging part of the presentembodiment.

FIG. 14B is a schematic drawing illustrating an example of theinterference fringe imaged by the imaging part of the presentembodiment.

DESCRIPTION OF EMBODIMENTS

A description of 2D-Structured Illumination Microscopy (2D-SIM) and3D-Structured Illumination Microscopy (3D-SIM) is given below, precedingthe description of the present invention.

While generally, with a fluorescence microscope, the fluorescencedistribution of a specimen containing a fluorescent substance isobserved, with 2D-SIM, a moiré is formed from the distribution offluorescence in the specimen and the distribution of structuredillumination by illuminating a specimen using an interference fringefrom the interference of two light fluxes (structured illumination).Then, by obtaining and demodulating this moiré image (modulated image),a high-resolution specimen image relating to the structure of thespecimen in the direction horizontal to the specimen plane (thedirection perpendicular to the optical axis) can be acquired.

Meanwhile, with 3D-SIM, because an interference fringe in the opticalaxis direction can be formed by illuminating the specimen using aninterference fringe from the interference of three light fluxes(structured illumination), a moiré can also be generated relating to thestructure of the specimen in the optical axis direction. This allows ahigh-resolution specimen image relating to the structure of the specimenin the optical axis direction to be acquired.

In the description of the embodiment below, 3D-SIM is used as an examplein description, but is also applicable to 2D-SIM.

[First Embodiment]

The first embodiment of the present invention is described below, withreference to drawings.

FIG. 1 is a simplified drawing illustrating an observation apparatus 1(structured illumination microscope) according to the first embodimentof the present invention. The observation apparatus 1 of the presentembodiment is, for example, a microscope apparatus for observing aspecimen SP that is a cell of a living body or the like.

In the observation apparatus 1, an illumination apparatus 10 and animaging part 210 are provided. The illumination apparatus 10 illuminatesthe specimen SP with an interference fringe. The imaging part 210 imagesa fluorescent image of the specimen SP modulated by an interferencefringe formed by an interference optical system 200, which is describedhereinafter.

The illumination apparatus 10 forms an interference fringe on apredetermined illumination region LA. The specimen SP is disposed on orin the vicinity of the illumination region LA. That is, the illuminationapparatus 10 forms an interference fringe on the specimen SP, along withilluminating the specimen SP.

In the illumination apparatus 10 according to the present embodiment, alight source apparatus 100, a light modulating part 120, a drivecontroller 160, and an interference optical system 200 are provided.

The light source apparatus 100 contains a light source 101, and isconfigured to emit a laser light at the light modulating part 120. Thelight modulating part 120 diffracts the entering light into a pluralityof orders.

The interference optical system 200 generates an interference fringe bymaking a plurality of diffracted light (branched light) diffracted bythe light modulating part 120 interfere with each other. Also, theinterference optical system 200 forms an image of the fluorescent imageof the specimen SP modulated by the interference fringe on the imagingplane of the imaging part 210. The drive controller 160 drives the lightmodulating part 120 and controls the phase, direction, and pitch of theinterference fringe. That is, the drive controller 160 is an example ofa control apparatus (controller) for applying drive voltage to the lightmodulating part 120. The drive controller 160 is configured from, forexample, a power source apparatus and a computer or the like.

The light source apparatus 100 according to the present embodimentincludes a light source 101, a light guide member 102, and a collimator104. The light source 101 includes a light generating element(one-dimensional light source) such as a laser diode or the like, andintroduces laser light to the light guide member 102. The light guidemember 102 includes, for example, an optical fiber, and guides theentering light from the light source 101 to the collimator 104. Notethat the exit end plane out of which the light from the light guidemember 102 exits acts as a two-dimensional light source. The collimator104 makes the light entering from the light source via the light guidemember 102 into parallel light.

The interference optical system 200 according to the present embodimentincludes a plurality of lens members. Specifically, in the interferenceoptical system 200, a polarized beam splitter 201, a lens group 202, amask 203, a dichroic mirror 204, a ½ wavelength plate 205, a filter 206,and a filter 207 are provided.

The polarized beam splitter 201 is disposed on the optical path betweenthe collimator 104, the light modulating part 120, and the lens 202-1.The polarized beam splitter 201 passes through light that is polarizedin the X direction and reflects light that is polarized in the Ydirection. The polarized beam splitter 201, along with directing thelight exiting from the collimator 104 to the light modulating part 120,directs one part of the light reflected by the light modulating part 120to the lens 202-1. Here, the light exiting from the collimator 104 canbe made into linear polarized light in the X direction. This allows theamount of light lost by the polarizing beam splitter 201 to be reduced.The operation of the polarizing beam splitter 201 is describedhereinafter.

The mask 203 is disposed on the optical path between the lens 202-1 andthe lens 202-2, and allows at least one part of the light exiting fromthe polarized beam splitter 201 to pass through. The mask 203 is in aplate shape, and is installed substantially perpendicular relative tothe optical axis AX1.

The ½ wavelength plate 205 is disposed on the optical path between thelens 202-1 and the lens 202-2, and changes the polarization condition ofthe light exiting from the polarized beam splitter 201 at each directionof the structured illumination. Specifically, the ½ wavelength plate 205converts the polarization condition of the light exiting from thepolarized beam splitter 201 to S polarized light relative to theentrance plane of the light in the illumination region LA. Note that the½ wavelength plate 205 may be installed anywhere as long as it isbetween the polarized beam splitter 201 and the object lens 202-4.

Lens 202-1 to 5 are included in the lens group 202. At least one fromamong lens 202-1 to 202-5 contains a lens member of a shape that isrotationally symmetrical around a predetermined axis of symmetry. Amongthese, the object lens 202-4 is a so-called object lens. Examples ofthis lens member include a spherical lens or an aspherical lens. In thepresent embodiment, the axis of symmetry of the lenses 202-1 to 202-3,which are lens members of a rotationally symmetrical shape, isappropriately called the optical axis AX1 of the interference opticalsystem. The lens 202-1 forms an optical surface OS1. The optical surfaceOS1 is a conjugate plane of the rear focal point plane (pupil plane) ofthe object lens 202-4, and is a so-called pupil conjugate plane.

The dichroic mirror 204 is a reflection transmission member that has adifferent reflectivity or transmissivity according to the wavelength ofthe light. The dichroic mirror 204 is disposed on the optical path ofthe lens 202-3, the object lens 202-4, and the lens 202-5, and has aproperty such that at least one part of the light entering from the lens202-3 is reflected to the direction of the object lens 202-4, and atleast one part of the light entering from the object lens 202-4 istransmitted to the direction of the lens 202-5.

The filter 206 is disposed on the optical path between the lens 202-3and the dichroic mirror 204, and transmits only excited light.

The filter 207 is disposed on the optical path between the dichroicmirror 204 and the lens 202-5, and transmits only fluorescence withouttransmitting excited light.

The lens 202-2 forms an optical surface 0S2 with the lens 202-1, whichsurface is optically conjugate to the light modulating part 120. Acenter image of the light modulating part 120 that is illuminated by thelight from the light source apparatus 100 is formed on the opticalsurface OS2. The lens 202-3 and the object lens 202-4 form an opticalsurface OS3 that is optically conjugate to the optical surface OS2.Because the optical surface OS2 is optically conjugate to the lightmodulating part 120, the optical surface OS3 is optically conjugate tothe light modulating part 120. The illumination region LA of theillumination apparatus 10 is set on the optical surface OS3 or in thevicinity of the optical surface OS3 so that the focus accuracy of theinterference fringe formed on the specimen SP is within a permissiblerange.

The lens 202-5 forms an optical surface OS4 that is optically conjugateto the illumination region LA. The optical surface OS4 corresponds tothe image plane when the illumination region LA is the object plane. Animage of the illuminated specimen SP is formed on the optical surfaceOS4. In the present embodiment, the object lens 202-4 and the lens 202-5each include a lens member of a shape that is rotationally symmetricalaround a predetermined axis of symmetry, and the axis of symmetry iscalled the optical axis AX2 of the interference optical system 200.Also, the optical axis AX2 is set practically perpendicular relative tothe part of the optical axis AX1 from the lens 202-1 to 202-3 in theinterference optical system 200. The plane through which the light fromthe specimen SP enters in the dichroic mirror 204 is slanted relative toeach of the optical axis AX1 of the interference optical system 200 andthe optical axis AX2 of the interference optical system 200, for exampleforming an angle of 45°. The light from the illuminated specimen SPenters the optical surface OS4 via the object lens 202-4, the dichroicmirror 204, and the lens 202-5.

The imaging part 210 according to the present embodiment includes animaging element 211 and an imaging controller 212. The imaging element211 includes an image sensor such as a CCD sensor, CMOS sensor, or thelike. The imaging element 211 includes a light receiving plane on whicha plurality of photodiodes is arranged, and a read circuit for reading asignal from the plurality of photodiodes. The light receiving plane ofthe imaging element 211 is disposed on the optical surface OS4, which isoptically conjugate to the illumination region LA on which the specimenSP is disposed. The light receiving plane of the imaging element 211 maybe offset from the optical surface OS4 within a range of focal depth inthe direction of the optical axis AX2 of the interference optical system200. The imaging controller 212 controls the read circuit of the imagingelement 211, and along with controlling imaging timing and the like,performs calculations for generating an image of the specimen SP basedon the signals from the read circuit.

The light modulating part 120 is described next in greater detail, withreference to FIG. 2A and 2B.

FIG. 2A is a schematic diagram illustrating an example of theconfiguration of the light modulating part 120 of the presentembodiment. The light modulator 120 is provided with a spatial lightmodulator (SLM) that uses ferroelectric liquid crystal (FLC). The liquidcrystal molecules of the ferroelectric liquid crystals have spontaneouspolarization and a layer structure. The light modulating part 120actualizes hastened switching of structured illumination by using as adiffraction grating a spatial light modulator that uses suchferroelectric liquid crystals. That is, the light modulating part 120 isan example of a branching member for branching light from the lightsource into a plurality of branched light. The light modulating part 120is provided with a plurality of pixels Px disposed in a grid shape onthe XY plane, which acts as a liquid crystal panel using ferroelectricliquid crystals, as illustrated in FIG. 2A. The light modulating part120 can change the direction, phase, and pitch of the interferencefringe occurring on the optical surface OS3 by changing the phase of theentering light via the voltage applied to the pixels Px.

The light modulating part 120 has a structure wherein a first electrodesubstrate 121 (first substrate) is stacked on a second electrodesubstrate 124 in the Z axis direction, as illustrated in FIG. 2B. Thefirst electrode substrate 121 is composed of, for example, a glasssubstrate, and has a pixel electrode 122 formed of, for example,silicon, on the surface thereof. There may be a circuit of, for example,TFT or the like, which is not shown, on the first electrode substrate121. The second electrode substrate 124 is composed of, for example, aglass substrate, and has a transparent electrode 123 on the surfacethereof. The light modulating part 120 applies a positive potentialvoltage (for example, voltage V1) or a negative potential voltage (forexample, voltage −V1) to the pixel electrode 122 of the first electrodesubstrate 121, where the electrical potential of the transparentelectrode 123 is the reference electrical potential.

Below, for the sake of convenience in description, the case where in apositive potential voltage (for example, voltage V1) is being applied toeach pixel electrode 122 will be called the “white state”, and for thesake of convenience in description, the case wherein a negativepotential voltage (for example, voltage −V1) is being applied will becalled the “black state”. As is described hereinafter, the lightmodulating part 120 can change the phase of the reflected light at eachpixel by applying a positive potential voltage or by applying a negativepotential voltage to each pixel electrode 122.

The configuration of the light modulating part 120 is described infurther detail. As for the spatial light modulator provided in the lightmodulating part 120, the molecules of the ferroelectric liquid crystalsare distributed in parallel within the XY plane, and the orientation ofthe molecules changes to two states according to the voltage appliedthereto. These two states correspond to the states of the “white state”and the “black state” described earlier. The molecules of theferroelectric liquid crystals have a long axis and a short axis. Becausethe refractive index of the ferroelectric liquid crystal molecules isdifferent in the long axis direction and the short axis direction, adifferent phase difference can be imparted to the light when linearpolarized light enters in the long axis direction and enters in theshort axis direction. That is, the spatial light modulator composing thelight modulating part 120 functions as a wavelength plate that canswitch its optical axis direction in two states based on the appliedvoltage. Also, because the spatial light modulator uses ferroelectricliquid crystals, the speed of the change in the refractive indexrelative to the change in voltage can be improved compared to whenusing, for example, nematic liquid crystals. For example, the opticalaxis direction can be switched to two states in microseconds.

The phase difference ΔΦ imparted by the spatial light modulator isexpressed by the following equation (1), where wavelength is λ, thedifference in refractive index of the long axis and the short axis ofthe molecule is Δn, and the element thickness is d.

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 1} \right\rbrack & \; \\{\mspace{315mu}{{\Delta\phi} = {\frac{2\pi}{\lambda}\Delta\;{nd}}}} & (1)\end{matrix}$

Here, in the case of a reflective spatial light modulator, d is replacedwith 2d. By setting so that the phase difference ΔΦ=π, the spatial lightmodulator functions as a λ/2 plate.

By utilizing this property, the spatial light modulator that usesferroelectric liquid crystals can be used as an amplitude-type or aphase-type diffraction grating. A specific example of the lightmodulating part 120 changing the phase of the entering light isdescribed with reference to FIG. 3A to 3C.

FIGS. 3A to 3C are schematic drawings illustrating an example of thephase difference of the light changed by the light modulating part 120of the present embodiment. As illustrated in FIG. 3A, a light L1, whichis generated by the power source 101 and passes through the light guidemember 102, the collimator 104 and the polarizing beam splitter 201,enters the light modulating part 120. The light modulating part 120modulates the phase of the entering light L1 via the white state or theblack state, and reflects a light L2. The polarizing beam splitter 201reflects only light in a specific polarized state from among theentering light L2 as a light L3. The reflected light L3 is directed tothe object lens 202-4 via the lens 202-1 and mask 203.

The modulation of the light by the light modulating part 120 isdescribed here in detail. FIG. 3B is a conceptual drawing of the pixelsPx in the light modulating part 120. In FIG. 3B, the straight linerepresents the long axis direction of the liquid crystal molecule. Theliquid crystal molecule shows a different orientation in the white stateand in the black state, as described above. Let the difference in anglebetween the two directions of the liquid crystal molecule be θ1; thedirection of the bisector of θ1 is defined as polarizing axis AX. Asillustrated in FIG. 3C(C-1), a case wherein the polarization state ofthe light L1 entering the light modulating part 120 is parallel to thepolarizing axis AXp, that is, a case wherein the angle formed betweenthe polarization direction of the light L1 and the polarizing axis AXp(angle θ0) is 0 (zero) [degrees] is described below.

Also, let the phase difference ΔΦ=π. At this time, the light modulatingpart 120 in the white state rotates the polarization direction of theentering light by angle θ1 in the counterclockwise direction relative tothe polarization axis AXp. Also, the light modulating part 120, when inthe black state, rotates the polarization direction of the enteringlight by angle θ1 in the clockwise direction relative to thepolarization axis AXp (see FIG. 3C(C-2)). That is, the angulardifference between the light L2 in the white state and the light L2 inthe black state is made angle 201 by the light modulating part 120.Here, the polarized light is vector decomposed, and an X polarized lightcomponent and a Y polarized light component are considered. The Ypolarized light component has a phase difference π [rad] at a pixel inthe white state and in the black state. That is, for the Y polarizedlight component, the light modulating part 120 is seen as a diffractiongrating with phase difference π [rad]. Thus, ±1st order diffractedlight, as well as high-order diffracted light is generated by the lightmodulating part 120. Meanwhile, for the X polarized light component, thephase difference at a pixel in the white state and the black state iszero. In this case, only 0th-order diffracted light is generated asdiffracted light from the light modulating part 120. The polarizing beamsplitter 201 reflects only the Y polarized light component from theentering light L2, and directs it to the lens 202-1. Thus, concerningthe light L2 in the white state, light of angle θ2 corresponding to theY component of the light of angle θ1 is selected, and concerning thelight L2 in the black state, light of angle −θ2 corresponding to the Ycomponent of the light of angle −θ1 is selected (see FIG. 3C(C-3)).Because of this, in the white state and the black state, the phasedifference of the polarization state of the light L3 becomes π [rad]. Asa result, just the light with equal polarization direction and phasedifference π [rad] can be extracted from the diffracted light generatedfrom the light modulating part 120. Polarization control is made easybecause the polarization of the diffracted light is all the same.Specifically, the polarization direction of the structured illuminationcan be made into S polarized light relative to the entrance plane of thelight in the illumination region LA in order to heighten the contrast ofthe interference fringe, but this polarization control is also madeeasy.

[Generation of Structured Illumination Light]

As described above, the light modulating part 120 modulates the enteringlight L1, and can function as a phase-type diffraction grating.Structured illumination can be generated by making the diffracted lightexiting from the light modulating part 120 interfere with each other.The light modulating part 120 generates diffracted light including +(positive) 1st order diffracted light, 0th order diffracted light, and −(negative) 1st order diffracted light. In the description below, thedirection of each diffracted light relative to the optical axis AX1 ofthe interference optical system 200 is appropriately named thediffraction direction.

Returning to the description of FIG. 1, the 0th order diffracted lightdiffracted by the light modulating part 120 is condensed by the lens202-1 onto a point A0 on the optical surface OS1 established by itsentrance angle into the lens 202-1. The +1st order diffracted lightdiffracted by the light modulating part 120 is condensed by the lens202-1 onto a point A1 on the optical surface OS1 established by itsentrance angle into the lens 202-1. The −1st order diffracted lightdiffracted by the light modulating part 120 is condensed by the lens202-1 onto a point A2 on the optical surface OS1 established by itsentrance angle into the lens 202-1. The mask 203 is installed in thevicinity of the condensing position of each diffracted light, and isconfigured to transmit the 0th order diffracted light, the +1st orderdiffracted light, and the −1st order diffracted light, and block otherdiffracted light.

Each diffracted light of the 0th order diffracted light and the ±1storder diffracted light passes through the mask 203 and enters the lens202-2. Each diffracted light is condensed to their respective positionson the rear side focal point plane (pupil plane) of the object lens202-4 by the lens 202-2 and 202-3. Each light exiting from theirrespective positions on the rear side focal point plane (pupil plane) ofthe object lens 202-4 becomes parallel light fluxes with differingangles, and exit toward the specimen SP from the object lens 202-4. Eachdiffracted light that has become a parallel light flux exiting from theobject lens 202-4 interferes over the specimen SP disposed on theillumination region LA.

In this manner, each diffracted light interferes over the specimen SP,and an interference fringe is formed on the specimen SP.

The phase of the interference fringe from the interference of eachdiffracted light becomes a phase according to the phase difference ofeach diffracted light in the illumination region LA. In other words, bycontrolling the phase difference of each diffracted light, the phase ofthe interference fringe in the illumination region LA can be controlled.

[Image Demodulation Based on the Interference Fringe]

The imaging part 210 (demodulating part) images the specimen image(modulated image, moiré image) modulated by the interference fringe, andby demodulating the imaged moiré image, acquires a high-resolutionimage. The method for the imaging part 210 demodulating the moiré imageis described below. As for the demodulation method, for example, themethod described in U.S. Pat. No. 8,115,806 may be used, but is notlimited to this method. An example of the demodulation method isdescribed below. First, for the sake of simplicity, it is describedusing the demodulation method for 2D-SIM as an example.

In an optical system with point image strength distribution Pr (x), thespecimen image obtained when providing illumination of a sinusoidalshape having a singular spatial frequency component K to a specimenhaving fluorescence density distribution Or (x) can be expressed as

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 2} \right\rbrack & \; \\{\mspace{155mu}{{I_{r}(x)} = {\sum\limits_{l}{{m_{l}\left( {{O_{r}(x)}{\exp\left( {{ilKx} + {i\;\phi}} \right)}} \right)}*{P_{r}(x)}}}}} & (2)\end{matrix}$

Here, Φ is the phase of the structured illumination.

Here, l=−1, 0, and l, and ml is the modulation amplitude of theillumination light. The l=0 component is a 0th order component that doesnot receive modulation from the structured illumination, and the l=−1,1components are respectively the ±1st order components (moiré) havingreceived modulation. In equation (2), the symbol * representsconvolutional integration. Below, amounts in real space are given thesubscript r, and amounts in frequency space are given the subscript k.Fourier transforming the equation (2) and writing in frequency spacegives

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 3} \right\rbrack & \; \\{\mspace{155mu}{{I_{k}(k)} = {\sum\limits_{l}{m_{l}{\exp\left( {{il}\;\phi} \right)}{O_{k}\left( {k + {lK}} \right)}{P_{k}(k)}}}}} & (3)\end{matrix}$

In equation (3), the Fourier transform of Pr (x), which is Pk (k),represents the optical transfer function (OTF).

Ok (k−K) and Ok (k+K) corresponding to l=−1 and 1 in equation (3) denotethat the spatial frequency component of the specimen is offset by thespatial frequency component K of the structured illumination. That is,even with an optical system that can only obtain through to a spatialfrequency component k, a higher spatial frequency component of thespecimen can be obtained. Because of this, the period of theinterference fringe can be made as short as possible within the rangethat an image can be formed by the optical system.

At this point, by imaging while offsetting the fringe pattern of theinterference fringe, N images with the same spatial frequency componentand modulation amplitude with only a different phase Φ can be obtained.The jth image signal strength lkj (k) is, with Φj as the structuredillumination phase of the jth image,

$\begin{matrix}\left\lbrack {{Equation}\mspace{14mu} 4} \right\rbrack & \; \\{\mspace{155mu}{{I_{kj}(k)} = {\sum\limits_{l}{m_{l}{\exp\left( {{il}\;\phi_{j}} \right)}{O_{k}\left( {k + {lK}} \right)}{P_{k}(k)}}}}} & (4)\end{matrix}$

That is, an N number of equations can be acquired from equation (4).Here, because Ok(k+lK), (l=−1,0,1) are unknown numbers in theseequations, these equations can be solved if N≥3.

Here, because the detectable range of Pk (k) of the optical system whenproviding an illumination with no strength distribution is k=−2NA/λ to2NA/λ, relating to the optical wavelength λ and the NA of the objectlens, the Ok(k+lK) obtained above in relation to l=−1, 0, and 1 includesthe information k=−2NA/λ−K to 2NA/λ−K, k=2NA/λ to 2NA/λ, and k=−2NA/λ+Kto 2NA/λ+K. Thus, because Ok(k+lK) on the whole includes informationfrom k=−2NA/λ−K to 2NA/λ+K, by redefining this as Ok (k), performing areverse Fourier transform and returning to the information in real space(an image of Or (x) of the specimen), a microscope image with highresolution can be acquired. That is, the imaging part 210 acquires asuper-resolution effect by performing image demodulation via theaforementioned calculations.

The image acquired as a result has a high resolution in only theone-dimensional direction to which spatial modulation was applied.Further, by changing the direction in which spatial modulation isimplemented to at least two directions, and by implementing the sametreatment as in one-dimension to each direction, the imaging part 210can acquire a microscope image with an isotropically high resolution intwo-dimension directions.

Note that the imaging part 210 can also acquire a high-resolutionmicroscope image by configuring simultaneous equations via theleast-squared method relative to the spatially modulated image andsolving them.

As described above, in 2D-SIM, a specimen image with high resolution ina one-dimension direction can be obtained by changing the phase of theinterference fringe and obtaining at least three images in aone-dimension direction.

Above, a demodulation method of 2D-SIM was described, but in 3D-SIM,because an interference fringe from the interference of 3 light fluxesis used, components that exist in the obtained image are fivecomponents, which are a 0th order component which does not receivemodulation, the ±2nd order components that act as the super-resolutioncomponents in the one-dimension direction in the plane of the specimen,and the ±1st order components that act as the super-resolutioncomponents in the optical axis direction. Thus, in 3D-SIM, because thereare five unknown numbers, by obtaining at least five images, the imagecan be reconstructed in the same manner as described above for 2D-SIM.In 3D-SIM, in addition to the direction of the plane, super-resolutionobservation can be actualized in the optical axis direction.

[Example of Interference Fringe]

An example of the interference fringe formed by the interference ofdiffracted light controlled by the drive controller 160 is describednext, with reference to FIG. 4. As described above, when the +(positive) 1st order diffracted light, 0th order diffracted light, and −(negative) 1st order diffracted light are made to interfere on theoptical surface OS3, by making the pattern of the interference fringe(henceforth fringe pattern) into five patterns, a super-resolutioneffect can be acquired in the planar direction of the optical surfaceOS3 and in the direction of the optical axis AX2, that is, it becomes3D-SIM. Each of the five patterns have the same direction and pitch asone another, and have a fringe pattern of differing phases. Note that inorder to acquire a super-resolution effect, from among interferencefringes formed by the interference of diffracted light from N patterns,the phase difference of neighboring fringe patterns can be made 2π/N[rad]. For example, in a case wherein there are five patterns of fringepatterns as described above, the phase difference of neighboring fringepatterns can be 2π/5 [rad]. However, for only 2D-SIM, it can be made π/3[rad]. Here, neighboring fringe patterns denote fringe patterns that areneighboring when fringe patterns are lined up so those with the smallestphase difference are next to each other. Note that considering theperiodicity of the phase, the phase difference may be 2π (m+1/N) [rad]where m is an integer, but because the phase difference of all of thesecan be calculated to be 2π [rad] or less, the phase difference isdescribed below as 2π [rad] or less.

Also, while the phrase “fringe pattern” was used in description above,the phase of the fringe pattern denotes the phase of the interferencefringe, the direction of the fringe pattern denotes the direction of theinterference fringe, and the pitch of the fringe pattern refers to thepitch of the interference fringe. This is the same for descriptionshereafter.

FIG. 4 is a schematic diagram illustrating an example of the voltagepattern applied by the drive controller 160 of the present embodiment.The drive controller 160 causes a fringe pattern of diffracted light onthe optical surface OS3 by controlling the voltage pattern applied toeach pixel Px of the light modulating part 120. Specifically, byapplying the voltage patterns A₁ to A₅ (image generation voltagepatterns) illustrated in FIG. 4 to the light modulating part 120, thedrive controller 160 generates fringe patterns according to thesevoltage patterns. In FIG. 4, portions illustrated as a white color(white portions) denote the white state described above, in which apositive potential voltage (for example, voltage V1) is applied.Meanwhile, in FIG. 4, portions illustrated as a black color (blackportions) denote the black state described above, in which a negativepotential voltage (for example, voltage − (negative) V1) is applied.Note that a negative potential voltage may be applied to the whiteportions, and a positive potential voltage may be applied to the blackportions. The voltage patterns in this example all have a voltagepattern for generating a fringe pattern with fringe pitch 5.3 [pixel]and fringe direction 2 [degrees]. Note that as the method for measuringangles, the 0 o'clock direction of a clock is defined as the angle 0[degrees], and the angle is defined to increase clockwise. However, theposition of the angle 0 [degrees] can be changed as appropriate. In thiscase, the fringe direction is changed accordingly. Here, the voltagepattern ratio of white portions and black portions is set as 1:1.Sequences other than this voltage pattern are described hereinafter.

These voltage patterns A₁ to A₅ are configured so the phase differenceof the interference fringe formed by each neighboring voltage pattern is2π/N [rad]. For example, the voltage pattern A₁ and the voltage patternA₂ are configured so the phase difference of the interference fringeformed by each of them is 2π/5 [rad]. Also, the voltage pattern A₂ andthe voltage pattern A₃ are configured so the phase difference of theinterference fringe formed by each of them is 2π/5 [rad].

The drive controller 160 makes it possible to acquire a super-resolutioneffect in the direction of the plane of the optical surface OS3 and inthe direction of the optical axis AX2 by sequentially generating thesevoltage patterns A₁ to A₅.

Here, when the total sum of the time of the phase difference between thefirst electrode substrate 121 and the second electrode substrate 124 ofthe light modulating part 120 is not zero, there exists a direct currentcomponent in the voltage between the first electrode substrate 121 andthe second electrode substrate 124. When there is a bias in the directcurrent component of this voltage, due to the interior ions in theferroelectric liquid crystals being pulled to one direction, aphenomenon called burn-in may occur, wherein they do not change toanother stable state, even when applying a voltage to the liquid crystalelement. Whether or not burn-in occurs between the first electrodesubstrate 121 and the second electrode substrate 124, and if burn-inoccurs, the degree to which it does, is determined by the direct currentcomponent of the voltage applied between the electrode substrates. Thus,burn-in can be prevented by applying a reverse voltage so as to offsetthe direct current component of the voltage applied between theelectrode substrates. Specifically, when generating the fringe patterndescribed above, an illumination period for illuminating light which isstructured illumination and a non-illumination period for notilluminating light which is structured illumination are provided. In thenon-illumination period, the light is, for example, blocked by ashutter, which is not shown. The drive controller 160 can preventburn-in by applying a voltage that is reverse of the illumination periodto the light modulating part 120 during the non-illumination period.That is, the drive controller 160 can prevent burn-in by applying areversed pattern calculated from the plurality of voltage patterns forirradiating light that is structured light to the light modulating part120.

Specifically, the drive controller 160 applies an inverse patternA_(inv) illustrated in FIG. 4 (burn-in prevention voltage pattern) tothe light modulating part 120. Here, the inverse pattern A_(inv) is thevoltage pattern wherein the electrical potential of the sum voltagepattern A_(sum) has been inversed. The sum voltage pattern A_(sum) isthe voltage pattern wherein the voltage value of each pixel in voltagepatterns A₁ to A₅ have been summed at each pixel, where the whiteportion of the voltage patterns A₁ to A₅ are +1 and the black portionsare −1. That is, the drive controller 160 applies the inverse patternA_(inv), which is the reverse voltage of the voltage pattern of the sumof the voltage patterns A₁ to A₅, to the light modulating part 120. Inthis manner, the drive controller 160 causes the total sum of the timeof the voltage applied to the light modulating part 120 to become zero,thereby offsetting the direct current component. Because of this, thedrive controller 160 is able to prevent burn-in of the light modulatingpart 120. At this time, the drive controller 160 applies a voltagepattern in, for example, the sequence illustrated in FIG. 5.

FIG. 5 is a schematic drawing illustrating an example of a sequence ofthe voltage pattern applied to the light modulating part 120 by thedrive controller 160 of the present embodiment. As illustrated in FIG.5, the drive controller 160 sequentially applies the voltage patterns A₁to A₅ and inverse pattern A_(inv) with the passage of time t. Here, letthe voltage patterns A₁ to A₅ be one period of voltage pattern. In otherwords, one period of voltage pattern is a plurality of fringe patternswith the same direction and pitch; in the 2D-SIM described above, it isthree patterns, and in 3D-SIM, it is five patterns. The drive controller160 applies voltage pattern A₁ from time t1 to t2. Then, the drivecontroller 160 applies voltage pattern A2 from time t2 to t3. In thismanner, the drive controller 160 sequentially applies voltage patternsA₃ to A₅ during time t3 to t6. Also, the drive controller 160 appliesthe inverse pattern A_(inv) during the non-irradiation period, wheretime t6 to t7 is the non-irradiation period. By sequentially applyingthe voltage patterns A₁ to A₅ and the inverse pattern A_(inv) asillustrated in FIG. 5, the drive controller 160 can prevent burn-in ofthe light modulating part 120.

Here, with reference to FIG. 6, the application sequence of the voltagepatterns according to the conventional method and the applicationsequence of the voltage patterns by the drive controller 160 of thepresent embodiment are compared and described.

FIG. 6 is a schematic drawing illustrating a comparative example of theapplication sequence of the voltage pattern. An application sequencethat differs from that of the present embodiment is illustrated in FIG.6(A). According to the application sequence illustrated in FIG. 6(A), aninverse pattern A₁′ relating to the voltage pattern A₁ is applied aftervoltage pattern A₁, that is, before the next voltage pattern A₂ isapplied. That is, according to the application sequence illustrated inFIG. 6(A), a voltage pattern and the inverse pattern relating to thisvoltage pattern are applied alternately. Here, if the time for applyingone voltage pattern is T, when alternately applying a voltage patternand its inverse pattern, the time required for applying one period ofvoltage pattern is the time from time t0A to time t10A, or in otherwords, 10T. The application sequence of the voltage pattern by the drivecontroller 160 of the present embodiment is illustrated in FIG. 6(B).According to the application sequence of the voltage pattern by thedrive controller 160, the inverse pattern A′ relating to the sum valueof voltage patterns A₁ to A₅ is applied after the voltage patterns A₁ toA₅ are applied. Here, the inverse pattern A′ in FIG. 6(B) is the inversepattern A_(inv) described above. That is, according to the applicationsequence of the voltage pattern by the drive controller 160, the inversepattern A′ is applied just once after the voltage patterns A₁ to A₅ areapplied. Here, when applying the inverse pattern just once afterapplying the voltage patterns, the time required for applying one periodof voltage patterns is from time t0B to time t6B, or in other words, 6T.Thus, according to the application sequence of the voltage patterns bythe drive controller 160, the time required for applying one period ofvoltage patterns can be reduced compared to the application sequenceillustrated in FIG. 6(A). In the case of this specific example, the timerequired for applying one period of voltage patterns can be reduced byapproximately 40 [%] with the application sequence of the voltagepatterns by the drive controller 160. This allows the time required forobservation by the observation apparatus 1 to be reduced by theillumination apparatus 10. In this manner, burn-in of the liquid crystalelement used as a spatial light modulator can be efficiently preventedby the drive controller 160 of the present embodiment. Also, forexample, when observing a living cell using the super-resolutionmicroscope according to the embodiment of the present invention,dynamics can be observed that are faster than conventional. Thus, thesuper-resolution microscope according to the embodiment of the presentinvention is, for example, suitable for observing living cells.

MODIFIED EXAMPLE 1

Note that until this point, although an example was described whereinthe drive controller 160 applied an inverse pattern relating to the sumvalue of the voltage patterns A₁ to A₅ just once after the applicationof the voltage patterns A₁ to A₅, it is not limited to such. The drivecontroller 160 may, for example, apply the voltage patterns and theinverse pattern according to the sequence illustrated in FIG. 7.

FIG. 7 is a schematic drawing illustrating a modified example of thesequence of the voltage patterns applied to the light modulating part120 by the drive controller 160 of the present embodiment. In thismodified example, the drive controller 160 applies an inverse patternA_(inv-1) (burn-in prevention voltage pattern) and an inverse patternA_(inv-2) (burn-in prevention voltage pattern) while the voltagepatterns A₁ to A₅ are applied. Specifically, the drive controller 160applies the inverse pattern A_(inv-1) after applying the voltagepatterns A₁ to A₃ and before applying the voltage pattern A₄. Then, thedrive controller 160 applies the inverse pattern A_(inv-2) afterapplying the voltage patterns A₄ to A₅ and before applying the voltagepattern A₁. Here, the inverse pattern A_(inv-1) and the inverse patternA_(inv-2) are inverse patterns generated by equally dividing the voltagevalue of A_(inv) illustrated in FIG. 5. Thus, this plurality of inversepatterns (here, inverse pattern A_(inv-1) and inverse pattern A_(inv-2))have the same electrical potential as the inverse pattern A_(inv)described above when summed. For example, the inverse pattern A_(inv-1)and the inverse pattern A_(inv-2) are each an inverse pattern of halfthe electrical potential of the inverse pattern A_(inv) described above.In this modified example, the drive controller 160 applies an inversepattern over a plurality of times in the application period of thevoltage patterns A₁ to A₅. Even when applying a plurality of inversepatterns over a plurality of times in this manner, the sum value ofvoltage of the inverse patterns applied in one application period ofvoltage patterns (for example, one period) is the same voltage as in theinverse pattern A_(inv) described above. This allows the direct currentcomponent from the voltage patterns A1 to A5 to be offset by inversepatterns, even when applying a plurality of inverse patterns over aplurality of times. Thus, even when applying a plurality of inversepatterns over a plurality of times, the time required for applying oneperiod of voltage patterns can be reduced compared to the applicationsequence illustrated in FIG. 6 (A) described above. This allows the timerequired for observation by the observation apparatus 1 to be reduced bythe illumination apparatus 10. In this manner, burn-in of the liquidcrystal element used as a spatial light modulator can be efficientlyprevented by the drive controller 160 of the present embodiment.

MODIFIED EXAMPLE 2

Although until this point, a case wherein the direction of the fringepattern is one direction, that is, a case of a fringe pattern whereinthe fringe pitch is 5.3 [pixel] and fringe direction is 2 [degrees] wasdescribed, it is not limited to such. Here, the observation apparatus 1can acquire an isotropic resolution in the plane of the optical surfaceOS3 by observing with a fringe pattern of a plurality of fringedirections rotated in a plurality of directions. For example, theobservation apparatus 1 can acquire an isotropic resolution in the planeof the optical surface OS3 by observing with a fringe pattern of threefringe directions. A modified example of the voltage pattern applied bythe drive controller 160 is described with reference to FIG. 8 and FIG.9.

FIG. 8 is a schematic diagram illustrating a first modified example ofthe voltage pattern applied by the drive controller 160 of the presentembodiment. By applying the voltage patterns B₁ to B₅ (image generationvoltage patterns) illustrated in FIG. 8 to the light modulating part120, the drive controller 160 generates fringe patterns according tothese voltage patterns. The voltage pattern in this modified example isa voltage pattern for generating a fringe pattern with fringe pitch 5.3[pixel] and fringe direction −58 [degrees]. Here, the drive controller160 applies the inverse pattern B_(inv) (burn-in prevention voltagepattern) illustrated in FIG. 8 to the light modulating part 120. Here,the inverse pattern B_(inv) is the voltage pattern wherein theelectrical potential of the sum voltage pattern B_(sum) has beeninversed.

The sum voltage pattern B_(sum) is the voltage pattern wherein thevoltage value of each pixel in voltage patterns B₁ to B₅ have beensummed at each pixel, where the white portions are +1 and the blackportions are −1.

That is, the drive controller 160 applies the inverse pattern, which isthe reverse voltage of the voltage pattern of the sum of the voltagepatterns B₁ to B₅, to the light modulating part 120. In this manner, thedrive controller 160 causes the total sum of the time of the voltageapplied to the light modulating part 120 to become zero, therebyoffsetting the direct current component.

FIG. 9 is a schematic diagram illustrating a second modified example ofthe voltage pattern applied by the drive controller 160 of the presentembodiment. By applying the voltage patterns C₁ to C₅ (image generationvoltage patterns) illustrated in FIG. 9 to the light modulating part120, the drive controller 160 generates fringe patterns according tothese voltage patterns. The voltage pattern in this modified example isa voltage pattern for generating a fringe pattern with fringe pitch 5.3[pixel] and fringe direction 62 [degrees]. Here, the drive controller160 applies the inverse pattern C_(inv) (burn-in prevention voltagepattern) illustrated in FIG. 9 to the light modulating part 120. Here,the inverse pattern C_(inv) is the voltage pattern wherein theelectrical potential of the sum voltage pattern C_(sum) has beeninversed.

The sum voltage pattern C_(sum) is the voltage pattern wherein thevoltage value of each pixel in voltage patterns C₁ to C₅ have beensummed at each pixel, where the white portions are +1 and the blackportions are −1.

That is, the drive controller 160 applies the inverse pattern which isthe reverse voltage of the voltage pattern of the sum of the voltagepatterns C₁ to C₅, to the light modulating part 120. In this manner, thedrive controller 160 causes the total sum of the time of the voltageapplied to the light modulating part 120 to become zero, therebyoffsetting the direct current component.

In the case of this modified example, the drive controller 160repeatedly applies the voltage pattern and the inverse pattern asillustrated in FIG. 6(B). Specifically, the drive controller 160 appliesthe inverse pattern A′ after applying the voltage patterns A₁ to A₅.Then, the drive controller 160 applies the inverse pattern B′ afterapplying the voltage patterns B₁ to B₅. Here, the inverse pattern B′ inFIG. 6(B) is the inverse pattern B_(inv) described above. Then, thedrive controller 160 applies the inverse pattern C′ after applying thevoltage patterns C₁ to C₅. Here, the inverse pattern C′ in FIG. 6(B) isthe inverse pattern C_(inv) described above. Even according to thismanner of application sequence, the time required for applying oneperiod of voltage patterns can be reduced compared to the applicationsequence illustrated in FIG. 6(A) above. This allows the time requiredfor observation by the observation apparatus 1 to be reduced by theillumination apparatus 10. In this manner, burn-in of the liquid crystalelement used as a spatial light modulator can be efficiently preventedby the drive controller 160 of the present embodiment.

Thus, the present invention is useful in the case of 3D-SIM, wherein anisotropic resolution can be acquired in the plane of the optical surfaceOS3 by observing via fringe patterns in three fringe directions.

Note that after displaying A₁ to A₅, B₁ to B₅, and C₁ to C₅, A′, B′, andC′ may be displayed. Also, while patterns with an equal ratio of whiteto black in FIGS. 4, 8, and 9 were described, the present method is notlimited to these. Below, the standardized ratio of white to black isdefined as the duty ratio. The duty ratio=0.5 when the ratio of white toblack is equal. A pattern with duty ratio 0.7 is illustrated in FIG. 10to FIG. 12. 0th order diffracted light can be generated more efficientlyby changing the duty ratio from 0.5.

FIG. 10 is a schematic drawing illustrating a third modified example ofthe voltage pattern applied by the drive controller of the presentembodiment. In FIG. 10, the direction and pitch are the same as FIG. 4,and only the duty is different. Five patterns D₁ to D₅ (image generationvoltage patterns) having different phases are illustrated. When the sumof each pattern is taken with black as −1 and white as +1, a two-valuevoltage pattern (pattern D_(sum)) composed of the minimum value −3 andthe maximum value −1 are acquired. The pattern D_(sum) is composed ofthe sum of one pattern D_(sum1) and two patterns D_(sum2). Morespecifically, as illustrated in FIG. 10, pattern D_(sum1) is a two-valuevoltage pattern composed of −1 and 1, and pattern D_(sum2) is aone-value voltage pattern composed of just −1. Thus, these inversepatterns are defined as pattern D_(inv1) (burn-in prevention voltagepattern) and D_(inv2) (burn-in prevention voltage pattern), and thesethree patterns are displayed only for the same amount of time as eachpattern of patterns D₁ to D₅. Here, the pattern D_(sum) is an example ofthe second voltage pattern. The pattern D_(sum) has one two-valuevoltage pattern (in this example, pattern D_(sum1)) having a firstvoltage value and a second voltage value, and a plurality of one-valuevoltage patterns (in this example, pattern D_(sum2)) having at leastonly one from among the first voltage value and the second voltagevalue.

FIG. 11 is a schematic diagram illustrating a fourth modified example ofthe voltage pattern applied by the drive controller of the presentembodiment. In FIG. 11, the direction and pitch are the same as FIG. 8,and only the duty is different. In FIG. 11, patterns E₁ to E₅ (imagegeneration voltage patterns) are five patterns having different phases.When the sum of each pattern is taken with black as −1 and white as +1,a two-value voltage pattern (pattern E_(sum)) composed of the minimumvalue −3 and the maximum value −1 are acquired. The pattern E_(sum) iscomposed of the sum of one pattern E_(sum1) and two patterns E_(sum2).More specifically, as illustrated in FIG. 11, pattern E_(sum1) is atwo-value voltage pattern composed of −1 and 1, and pattern E_(sum2) isa one-value voltage pattern composed of just −1. Thus, these inversepatterns E_(inv) (burn-in prevention voltage patterns) are composed ofpattern E_(inv1) and pattern E_(inv2).

FIG. 12 is a schematic drawing illustrating a fifth modified example ofthe voltage pattern applied by the drive controller of the presentembodiment. In FIG. 12, the direction and pitch are the same as FIG. 9,and only the duty is different. In FIG. 12, patterns F₁ to F₅ (imagegeneration voltage patterns) are five patterns having different phases.When the sum of each pattern is taken with black as −1 and white as +1,a two-value voltage pattern (pattern F_(sum)) composed of the minimumvalue −3 and the maximum value −1 are acquired. The pattern F_(sum) iscomposed of the sum of one pattern F_(sum1) and two patterns F_(sum2).More specifically, as illustrated in FIG. 12, pattern F_(sum1) is atwo-value voltage pattern composed of −1 and 1, and pattern F_(sum2) isa one-value voltage pattern composed of just −1. Thus, these inversepatterns F_(inv) (burn-in prevention voltage patterns) are composed ofpattern F_(inv1) and pattern F_(inv2).

Here, the number of patterns with which the inverse pattern is expressedis determined by the maximum or minimum value when the sum is taken. Ifthe absolute value of the maximum value and the minimum value are both1, one pattern is sufficient for the inverse pattern. This correspondsto what is shown in each figure of FIG. 4, FIG. 8, and FIG. 9. If theabsolute value of the maximum value or the minimum value exceeds 1,three patterns are necessary for the inverse pattern. This correspondsto what is shown in each figure of FIG. 10 to FIG. 12. The image displaysequence when executing 3D-SIM with the patterns illustrated in eachfigure of FIG. 10 to FIG. 12 is illustrated in FIG. 13.

FIG. 13 is a schematic diagram illustrating a modified example of theapplication sequence of the voltage pattern applied by the drivecontroller of the present embodiment. FIG. 13(A) is a case wherein aninverse pattern is displayed after each pattern, and FIG. 13(B) is acase using the present method. By using the present method, the numberof display patterns can be reduced to 4/5, allowing for theactualization of a 1.25× hastening.

The direction and pitch of the interference fringe generated by theillumination apparatus 10, that is, the measurement method of thefrequency vector, is described next with reference to FIG. 14A and 14B.

FIG. 14A and 14B are schematic drawings illustrating an example of theinterference fringe imaged by the imaging part 210 of the presentembodiment. As described above, the imaging part 210 images an image ofthe interference fringe generated on the optical surface OS3, and bydemodulating the imaged image, acquires a high-resolution image. At thistime, for example, the imaging controller 212 can measure the directionand pitch of the interference fringe generated on the optical surfaceOS3, that is, the frequency vector, based on the image of theinterference fringe generated on the imaged optical surface OS3. Forexample, as illustrated in FIG. 14B (B-1) to (B-3), even when thedirection of the interference fringe changes in three directions, theimaging controller 212 can measure the frequency vector.

For example, in FIG. 14A, the arrow a illustrates the direction of theinterference fringe, the arrow b illustrates the direction of thefrequency vector of the interference fringe, and the arrow c illustratesthe pitch of the interference fringe.

The direction and pitch of the interference fringe generated by theillumination apparatus 10, that is, the frequency vector, can also bemeasured in the following manner. The observation apparatus 1 can havedisposed the reflective surface of a mirror ML in place of the specimenSP in the position of the optical surface OS3. The imaging part 210images an image of the interference fringe reflected by the mirror ML.In this case, the dichroic mirror 204 can be replaced with a half mirror(not shown). At this time, for example, the imaging controller 212 canmeasure the direction and pitch of the interference fringe generated onthe optical surface OS3, that is, the frequency vector, based on theimage of the interference fringe generated on the imaged optical surfaceOS3.

Note that the direction (fringe direction) of the interference fringegenerated when the voltage patterns A1 to A5 are applied is 2 [degrees],but an offset of direction (deflection) relative to the desired value ispossible within 5 [degrees]. Also, this offset (deflection) ispreferable within 2 [degrees], and more preferably within 1 [degrees].This offset (deflection) is the same for the direction of theinterference fringe generated when applying voltage patterns B₁ to B₅,and for the direction of the interference fringe generated when applyingvoltage patterns C₁ to C₅, as for the direction of the interferencefringe caused when applying voltage patterns A₁ to A₅.

Note that in the above, the drive controller 160 was described asapplying a drive voltage applied to the light modulating part 120 to allof the pixels Px provided in the light modulating part 120, but it isnot limited to such. The drive controller 160 may, for example, apply adrive voltage to only pixels effective for structured illumination(effective pixels) from among all of the pixels Px provided in the lightmodulating part 120.

Note that in the above, an example was described wherein the voltageapplied to the light modulating part 120 by the drive controller 160 wasa voltage of two values, voltage V1 and voltage − (negative) V1, but itis not limited to such. The drive controller 160 may be a configurationthat applies voltage other than the two values described above to thelight modulating part 120. Further, the drive controller 160 wasdescribed as offsetting the direct current component with an inversepattern from the two values of voltage based on the calculated twovalues of voltage, but it is not limited to such. The drive controller160 should apply an inverse pattern that offsets the sum value of thevoltage patterns. For example, when the sum value of the voltagepatterns is a voltage value other than the two values described above,the drive controller 160 may offset the sum value with an inversepattern that is the reverse voltage of a value of voltage other than thetwo values. Also, the drive controller 160 may be a configuration thatoffsets the sum value of the voltage pattern by changing the applicationtime of the inverse pattern. For example, if the sum value of thevoltage patterns is double the voltage of the voltage value of theinverse pattern, the drive controller 160 offsets the sum value of thevoltage patterns by doubling the time that the inverse pattern isapplied. Also, the drive controller 160 was described as applying theinverse pattern every one period of voltage patterns, but it is notlimited to such. The drive controller 160 may apply an inverse patternthat is the sum value of a plurality of periods every plurality ofperiods of voltage patterns.

Above, an embodiment of the present invention was described in detailwith reference to diagrams, but the specific configuration is notlimited to this embodiment, and appropriate changes may be added withina range that does not deviate from the meaning of the present invention.The configurations described in each embodiment above may be combined.

Note that while in FIG. 1 and the like, the mask 203 is disposed on theoptical path between the lens 202-1 and the lens 202-2, it may bedisposed in any position between the polarizing beam splitter 201 andthe illumination region LA that does not cause the optical paths of theplurality of light fluxes diffracted by the light modulating part 120 tooverlap each other. For example, the mask 203 may be disposed in aposition in a plane that is optically conjugate to the optical surfaceOS1, or the vicinity thereof.

Note that the configuration of the light source apparatus 100 may bechanged as appropriate. In the embodiment above, the light sourceapparatus 100 is a part of the illumination apparatus 10, but at leastone part of the light source apparatus 100 may be an external apparatusto the illumination apparatus 10.

Note that each lens in FIG. 1 and the like are drawn as one member, butthe number of lens members of each lens may be one, or two or more. Theinterference optical system 200 may include a cut lens with one part ofthe rotationally symmetrical lens member cut, or it may include arotationally non-symmetrical free-form lens.

Note that a case wherein the light modulating part 120 was areflective-type was described in FIG. 1 and the like, but the lightmodulating part 120 may be a transmission-type.

Note that each part provided in each controller (drive controller 160,imaging controller 212) provided in the observation apparatus 1 of eachembodiment may be actualized with specialized hardware, or actualizedwith memory and a microprocessor.

Note that each controller provided in the observation apparatus 1 may beconfigured from a memory and a CPU (central calculation apparatus), andactualize the function thereof by loading a program for actualizing thefunction of each part provided to the display apparatus to the memoryand executing it.

Also, the processes of each part provided to the controller may beperformed by recording a program for actualizing the function of eachcontroller provided in the observation apparatus 1 on a readablerecording medium, loading the program recorded on the recording mediumonto a computer system, and executing it. Note that “computer system” asused here includes hardware such as an OS, peripheral devices, and thelike.

Also, if the “computer system” is using a WWW system, it includes thehome page provision environment (or the display environment).

Also, “computer readable recording medium” refers to recordingapparatuses such as portable media such as a flexible disk, amagneto-optical disk, ROM, CD-ROM, and the like, and hard disks and thelike that are contained in a computer system. Further, “computerreadable recording medium” also includes things that retain a programfor a given amount of time like a volatile memory inside a computersystem that acts as the server or client when, for a short time, aprogram is retained dynamically like a communication line when sending aprogram via a network such as the internet or a communication circuitsuch as a phone circuit. Also, the program may be for actualizing onepart of the functions described above, or may be further combined with aprogram for the functions described above recorded on a computer system.

In the embodiment, the structured illumination apparatus includes afirst substrate on which a plurality of pixel electrodes are provided, asecond substrate facing the first substrate, and ferroelectric liquidcrystals interposed between the first substrate and the secondsubstrate, and is provided with a branching member for branching lightfrom a light source into a plurality of branched light; an interferenceoptical system for illuminating a specimen via an interference fringegenerated by causing the plurality of branched light to interfere witheach other; a controller for applying a drive voltage of a drive voltagevalue shown by a voltage pattern to the ferroelectric liquid crystalsvia the plurality of pixel electrodes, wherein the voltage pattern isthe distribution of drive voltage values applied to every pixelelectrode; wherein a plurality of first voltage patterns withdistributions differing from each other and at least one second voltagepattern based on the sum value of the plurality of the first voltagepatterns is are included in the voltage pattern, and the controllersequentially applies the drive voltage of the drive voltage value shownby the plurality of the first voltage pattern and the second voltagepattern.

In the above embodiment, the first voltage pattern can be a two-valuevoltage pattern having a first voltage value of a positive electricalpotential, and a second voltage value having the same absolute value asthe first voltage value having a negative electrical potential.

In the above embodiment, the second voltage pattern can be a two-valuevoltage pattern having a first voltage value and a second voltage value.

In the above embodiment, the second voltage pattern can be a two-valuevoltage pattern having the first voltage value and the second voltagevalue, and it can have a plurality of one-value voltage patterns havingat least only one from among the first voltage value and the secondvoltage value.

In the above embodiment, the second voltage pattern can be generatedbased on the reverse voltage of the sum value of the drive voltage valueshown by the plurality of the first voltage patterns.

In the above embodiment, the pattern of the interference fringegenerated by the interference optical system is determined according tothe distribution of the voltage pattern and the plurality of the firstvoltage pattern may have a voltage pattern group composed of N of thefirst voltage pattern, and in the voltage pattern group, the N of thefirst voltage patterns can each be generated so the directions of thepatterns of the interference fringes match each other.

In the above embodiment, the N of the first voltage pattern can begenerated so the phase difference of the pattern of the neighboringinterference fringes is 2π/N.

In the above embodiment, the plurality of the first voltage pattern mayhave a plurality of the voltage pattern group, and the plurality of thevoltage pattern group may be generated so the directions of theinterference fringes are different from each other.

In the above embodiment, at least one of the second voltage pattern maybe generated based on the reverse voltage of the sum value of the drivevoltage value shown by the N of the first voltage pattern, and thecontroller may sequentially apply the drive voltage shown by each of theN of the first voltage patterns, and the drive voltage shown by thegenerated second voltage pattern.

In the above embodiment, the second voltage pattern may be generatedbased on the reverse voltage of the sum value of the drive voltagevalues shown by each of the first voltage patterns included in all ofthe voltage pattern groups, and the controller may sequentially applythe drive voltage of the drive voltage value shown by the plurality ofthe first voltage pattern included in the voltage pattern group forevery voltage pattern group, and apply the drive voltage shown by thesecond voltage pattern.

In a separate embodiment, a structured illumination microscope apparatusis provided with the structured illumination apparatus described in anyone of the embodiments, an imaging apparatus for imaging a modulatedimage of the specimen formed by forming an image of the observationlight from the specimen illuminated by the interference fringe andobtaining modulated image of the specimen, and a calculation apparatusfor demodulating the modulated image.

What is claimed is:
 1. A structured illumination microscope comprising:a spatial light modulator; an illumination optical system forilluminating a specimen with illumination light from the spatial lightmodulator; a first controller for applying a voltage pattern having apredetermined voltage value distribution to the spatial light modulator;an imaging element for generating an image of the specimen; and a secondcontroller for generating a calculated image using a plurality of theimages, wherein the first controller applies plural image generationvoltage patterns for generating the calculated image and at least oneburn-in prevention voltage pattern for preventing burn-in generated bythe plural image generation voltage patterns, and an entire period oftime in which the at least one burn-in prevention voltage pattern isapplied is shorter than an entire period of time in which the pluralimage generation voltage patterns are applied.
 2. The structuredillumination microscope according to claim 1, wherein the at least oneburn-in prevention voltage pattern is not used for generating thecalculated image.
 3. The structured illumination microscope according toclaim 1, wherein the at least one burn-in prevention voltage pattern iscalculated based on the plural image generation voltage patterns.
 4. Thestructured illumination microscope according to claim 1, wherein theillumination optical system illuminates the specimen with aninterference fringe, and the first controller applies the plural imagegeneration voltage patterns to change a phase and a direction of theinterference fringe, and applies the at least one burn-in preventionvoltage pattern before and after the change in direction of theinterference fringe.
 5. The structured illumination microscope accordingto claim 1, wherein the at least one burn-in prevention voltage patternis calculated based on a reverse voltage of a sum voltage pattern, whichis obtained by summing the plural image generation voltage patterns. 6.The structured illumination microscope according to claim 1, wherein thespatial light modulator includes a first substrate including a pluralityof pixel electrodes, a second substrate opposing the first substrate,and liquid crystals positioned between the first substrate and thesecond substrate, and the first controller applies the voltage patternto the spatial light modulator via the pixel electrodes.
 7. Thestructured illumination microscope according to claim 6, wherein theliquid crystals are ferroelectric liquid crystals.
 8. The structuredillumination microscope according to claim 1, wherein the imagegeneration voltage patterns are composed of a first voltage value havinga positive electric potential and a second voltage value having anegative electric potential with the same absolute value as the firstvoltage value.
 9. The structured illumination microscope according toclaim 1, wherein the illumination optical system illuminates thespecimen with an interference fringe, and the first controller appliesthe plural image generation voltage patterns to change a phase and adirection of the interference fringe, and applies the at least oneburn-in prevention voltage pattern after changing the direction of theinterference fringe at least twice.
 10. A structured illumination methodcomprising: (a) illuminating a specimen with illumination light from aspatial light modulator; (b) applying a voltage pattern having apredetermined voltage value distribution to the spatial light modulator;(c) generating an image of the specimen; and (d) generating a calculatedimage using a plurality of the images, wherein plural image generationvoltage patterns for generating the calculated image and at least oneburn-in prevention voltage pattern for preventing burn-in generated bythe plural image generation voltage patterns are applied in (b), and anentire period of time in which the at least one burn-in preventionvoltage pattern is applied is shorter than an entire period of time inwhich the plural image generation voltage patterns are applied.
 11. Anon-transitory computer readable medium storing a program for causing acomputer to execute the steps of: (a) illuminating a specimen withillumination light from a spatial light modulator; (b) applying avoltage pattern having a predetermined voltage value distribution to thespatial light modulator; (c) generating an image of the specimen; and(d) generating a calculated image using a plurality of the images,wherein plural image generation voltage patterns for generating thecalculated image and at least one burn-in prevention voltage pattern forpreventing burn-in generated by the plural image generation voltagepatterns are applied in (b), and an entire period of time in which theat least one burn-in prevention voltage pattern is applied is shorterthan an entire period of time in which the plural image generationvoltage patterns are applied.